Repository navigation
Fix stale request cache, auth resets, and download() error/session handling - #2994
Open
consciousmediacm-dev wants to merge 1 commit into
Open
consciousmediacm-dev wants to merge 1 commit into
consciousmediacm-dev wants to merge 1 commit into
Conversation
…ndling - cache_get(): replace the never-expiring lru_cache with a bounded cache whose entries expire after yf.config.network.cache_ttl seconds (default 60; 0 disables, None never expires). Error responses are not cached. Adds YfData.cache_clear(). - _make_request(): a 404 (delisted/mistyped ticker) no longer switches cookie strategy and wipes the crumb shared by all threads. - _get_cookie_basic(): mark the cookie as fetched, so fc.yahoo.com isn't re-requested on every call when the persistent cookie cache is unusable. - _set_session(): reset cookie/crumb when the session object changes. - download(): only replace the shared session when the caller passes one; request exceptions per call via PriceHistory._history(raise_errors=True) instead of toggling a config key nobody read (network.hide_exceptions), which would also have raced between threads. - Tickers.download(): drop dead assignment to an unused _history attribute. - Add offline regression tests (tests/test_reliability_fixes.py) and document cache_ttl. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01VxuCGhah3SL3ZWQrwXUQDR
Collaborator
|
Can these be separated into individual PRs? |
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Six reliability fixes, mostly affecting long-running processes and multi-ticker
download(). Each has an offline regression test intests/test_reliability_fixes.py.Bug Fix
1
YfData.cache_get()useslru_cachewith no expiry, so in a long-running process e.g.Market("US").summary,Search, and fundamentals return the first response forever Bounded cache with TTLyf.config.network.cache_ttl(default 60s;0disables,Nonenever expires). Error responses are not cached. AddsYfData.cache_clear(). Documented indoc/source/advanced/config.rst2 Any HTTP >= 400, including 404 for a delisted/mistyped ticker, toggles the cookie strategy and wipes the crumb shared by all threads (one bad ticker caused 7 requests in a mocked trace) 404 no longer triggers the cookie-strategy retry; 401 etc. still do
3
download()togglesconfig.network.hide_exceptions, a key nothing reads (debug.hide_exceptionsis the real one). Fixing just the name would race between threads_download_onecallsPriceHistory._history(raise_errors=True)per call; global config untouched. Publichistory(raise_errors=...)keeps its deprecation warning4 Every
download()withoutsession=creates a new session and swaps it into theYfDatasingleton, keeping the old crumb with an empty cookie jar Only swap when a session is passed;_set_sessionresets cookie/crumb when the session object changes5
Tickers.download()assigns results to an unused_historyattribute (with a{}fallback that would raise) Removed6 With the
basicstrategy,_get_cookie_basicnever setsself._cookie, sofc.yahoo.comis re-requested on every call whenever the persistent cookie cache is unusable (read-only cache dir, multiple Yahoo cookies) Setself._cookie = Trueafter a successful fetch, as the csrf path already doesBehaviour changes to note
Inside
download(), per-ticker history errors now raise into the collector as the original code intended. A soft failure such asauto_adjustfailing yields an empty frame plus one grouped error log for that ticker, instead of unadjusted data plus a duplicate per-ticker log line.Historical-range chart requests previously cached forever; they now refetch after
cache_ttl(setNoneto restore the old behaviour).Testing
New tests: 21 pass. Against unmodified
dev, 16 fail (they catch the bugs) and 5 pass (guards for behaviour that must not change, e.g. a 401 still retries).python -m unittest tests.test_reliability_fixes: OK.Offline suite (http, data, cookie/crumb degradation, download concurrency, utils, auth, live, screener, multi): 100 passed. 1 failure is environmental:
US/Easternmissing from the test machine's tzdata, identical without this change.Network-dependent suites couldn't reach Yahoo from the development machine; results were identical with and without this change.
ruff checkclean.Separately exercised against live Yahoo through a downstream wrapper (quotes, daily/intraday history, financials, info, bad-ticker handling): all passed.
Claude-Session: https://claude.ai/code/session_01VxuCGhah3SL3ZWQrwXUQDR